OPRUN-4607: Fix flags passed to test-experimental-e2e by tmshort · Pull Request #723 · openshift/operator-framework-operator-controller

tmshort · 2026-05-11T18:14:31Z

UPSTREAM: : Fix downstream e2e test invocation

Replace broken test-experimental-e2e target (test/experimental-e2e no
longer exists) with /bin/true so triggered jobs always succeed
Pass -timeout=60m to go test; the previous invocation relied on Go's
10m default which is too short for BoxcutterRuntime clusters
Set E2E_STEP_TIMEOUT=15m; BoxcutterRuntime applies resources through
sequential phases (CRD must reach Established before the deploy phase
starts), making installations slower than the upstream 5m default
Skip ~@CatalogdHA scenarios (require multiple catalogd replicas not
present in standard topology)
Skip ~@ProgressDeadline scenarios (require progressDeadlineMinutes < 10
but the OpenShift CRD enforces a minimum of 10)
Skip ~https://github.com/httpproxy scenarios (too disruptive to cluster networking)

Summary by CodeRabbit

Tests
- Experimental end-to-end test target is now disabled in automated runs and immediately succeeds, skipping downstream experimental checks.
- Main end-to-end test target now uses a longer 50-minute timeout and passes that timeout to the test runner to prevent hangs.
- The exported experimental E2E flags have been removed.

coderabbitai · 2026-05-11T18:14:45Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

Walkthrough

Makefile edits in openshift/Makefile: test-experimental-e2e was replaced with a no-op (/bin/true) and the exported DOWNSTREAM_EXPERIMENTAL_E2E_FLAGS was removed; DOWNSTREAM_E2E_TIMEOUT := 50m was added and test-e2e's go test invocation now includes -timeout=$(DOWNSTREAM_E2E_TIMEOUT) and is reformatted across lines.

Changes

Openshift Makefile — E2E targets

Layer / File(s)	Summary
Timeout variable and comment `openshift/Makefile`	Add `export DOWNSTREAM_E2E_TIMEOUT := 50m` and a comment noting the default timeout bump.
test-e2e: test invocation formatting & timeout `openshift/Makefile`	Refactor `test-e2e` recipe's `go test` invocation across multiple lines and add `-timeout=$(DOWNSTREAM_E2E_TIMEOUT)` while keeping `--godog.tags="$(DOWNSTREAM_GODOG_FLAGS)"` and `--k8s.cli=oc`.
test-experimental-e2e: no-op replacement & flag removal `openshift/Makefile`	Replace previous build/test steps for `test-experimental-e2e` with `/bin/true` and remove the exported `DOWNSTREAM_EXPERIMENTAL_E2E_FLAGS` variable.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 11 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title describes a specific fix to flags in test-experimental-e2e, but the actual main change is disabling that target entirely (runs /bin/true) and adding timeout to test-e2e instead.	Revise the title to accurately reflect the primary changes: disabling test-experimental-e2e and adding a 50m timeout to test-e2e, or clarify the intended scope.

✅ Passed checks (11 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	PR only modifies openshift/Makefile (build config). No Ginkgo test files or test names are changed, so the check is not applicable.
Test Structure And Quality	✅ Passed	PR modifies only openshift/Makefile. No Ginkgo test code files in test/ were changed. Custom check for test code quality is not applicable to this PR.
Microshift Test Compatibility	✅ Passed	This PR modifies openshift/Makefile only—adding a timeout flag to test-e2e and disabling test-experimental-e2e. No new Ginkgo tests are added. The check is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	PR only modifies openshift/Makefile (test execution targets), not test code. No new Ginkgo e2e tests are added, so custom check for SNO test compatibility is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR modifies only openshift/Makefile test infrastructure. No deployment manifests, operator code, controllers, or scheduling constraints are introduced.
Ote Binary Stdout Contract	✅ Passed	PR only modifies openshift/Makefile for test orchestration. The test/e2e code added is a Godog BDD test suite, not an OTE binary. OTE stdout contract check does not apply.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	This PR does not add new Ginkgo e2e tests. It only modifies openshift/Makefile to disable test-experimental-e2e and add timeout to test-e2e. Check applies only when new tests are added.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-ci · 2026-05-11T18:15:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~DOWNSTREAM_OWNERS~~ [tmshort]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tmshort · 2026-05-11T18:15:15Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-11T18:28:08Z

/test openshift-e2e-aws-techpreview

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openshift/Makefile`:
- Around line 47-48: Replace the short-circuit in the Makefile target
test-experimental-e2e (remove the "/bin/true" line) and restore the real
command(s) that run the experimental e2e suite so the target actually executes
tests; ensure the restored command invokes the same test runner used elsewhere
(e.g., the existing e2e script or binary) and passes the corrected flags/ENV
used for experimental runs so CI executes the suite instead of always
succeeding.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c1ad50ea-2428-4502-86f2-c79d425a54c2

📥 Commits

Reviewing files that changed from the base of the PR and between 0c7d0b1 and 7aa1838.

📒 Files selected for processing (1)

openshift/Makefile

tmshort · 2026-05-11T19:57:15Z

/test openshift-e2e-aws-techpreview

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openshift/Makefile`:
- Around line 46-49: The Go test command in the Makefile uses the misspelled
flag "-tieout" which should be "-timeout"; update the invocation (the go test
line that includes $(DOWNSTREAM_E2E_FLAGS) and $(DOWNSTREAM_E2E_TIMEOUT)) to use
"-timeout=$(DOWNSTREAM_E2E_TIMEOUT)" so the Go test timeout is applied and the
make test-e2e target no longer fails due to an unknown flag.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 83c75b64-475b-430d-885d-47b1d524fd27

📥 Commits

Reviewing files that changed from the base of the PR and between 7aa1838 and d36da45.

📒 Files selected for processing (1)

openshift/Makefile

tmshort · 2026-05-12T14:01:36Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-12T15:46:32Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-12T18:55:23Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-13T12:59:45Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-13T13:48:37Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-14T18:22:41Z

/test openshift-e2e-aws-techpreview

tmshort · 2026-05-14T18:22:50Z

/trest openshift-e2e-aws

tmshort · 2026-05-15T18:41:55Z

/test openshift-e2e-aws-techpreview

- Replace broken test-experimental-e2e target (test/experimental-e2e no longer exists) with /bin/true so triggered jobs always succeed - Pass -timeout=60m to go test; the previous invocation relied on Go's 10m default which is too short for BoxcutterRuntime clusters - Set E2E_STEP_TIMEOUT=15m; BoxcutterRuntime applies resources through sequential phases (CRD must reach Established before the deploy phase starts), making installations slower than the upstream 5m default - Skip ~@CatalogdHA scenarios (require multiple catalogd replicas not present in standard topology) - Skip ~@ProgressDeadline scenarios (require progressDeadlineMinutes < 10 but the OpenShift CRD enforces a minimum of 10) - Skip ~@httpproxy scenarios (too disruptive to cluster networking) Co-Authored-By: Claude Sonnet 4.6 <[email protected]> Signed-off-by: Todd Short <[email protected]>

openshift-ci-robot · 2026-05-19T02:27:04Z

@tmshort: This pull request references OPRUN-4607 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary by CodeRabbit

Tests

Experimental end-to-end test target is now disabled in automated runs and immediately succeeds, skipping downstream experimental checks.

Main end-to-end test target now uses a longer 50-minute timeout and passes that timeout to the test runner to prevent hangs.

The exported experimental E2E flags have been removed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

dtfranz · 2026-05-19T06:15:45Z

/lgtm

openshift-ci · 2026-05-19T06:28:05Z

@tmshort: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/openshift-e2e-aws-techpreview	`3fbff88`	link	false	`/test openshift-e2e-aws-techpreview`
ci/prow/e2e-aws-upgrade-ovn-single-node	`d1f5659`	link	false	`/test e2e-aws-upgrade-ovn-single-node`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

dtfranz · 2026-05-19T09:35:50Z

/verified by @dtfranz
/test e2e-aws-upgrade-ovn-single-node

openshift-ci-robot · 2026-05-19T09:35:54Z

@dtfranz: The /verified command must be used with one of the following actions: by, later, remove, or bypass. See https://docs.ci.openshift.org/docs/architecture/jira/#premerge-verification for more information.

Details

In response to this:

/verified
/test e2e-aws-upgrade-ovn-single-node

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci Bot requested review from fgiudici and perdasilva May 11, 2026 18:15

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 11, 2026

tmshort force-pushed the fix-experimental-e2e branch from 0c7d0b1 to 7aa1838 Compare May 11, 2026 18:26

coderabbitai Bot reviewed May 11, 2026

View reviewed changes

Comment thread openshift/Makefile

tmshort force-pushed the fix-experimental-e2e branch from 7aa1838 to d36da45 Compare May 11, 2026 19:52

coderabbitai Bot reviewed May 11, 2026

View reviewed changes

Comment thread openshift/Makefile

tmshort force-pushed the fix-experimental-e2e branch from d36da45 to c79c8d7 Compare May 12, 2026 13:33

tmshort force-pushed the fix-experimental-e2e branch from c79c8d7 to e073781 Compare May 12, 2026 15:46

tmshort force-pushed the fix-experimental-e2e branch from e073781 to 2687456 Compare May 12, 2026 18:51

tmshort force-pushed the fix-experimental-e2e branch from 2687456 to f17a6e4 Compare May 12, 2026 23:52

tmshort force-pushed the fix-experimental-e2e branch from 4df1f20 to c938bcf Compare May 14, 2026 14:22

tmshort force-pushed the fix-experimental-e2e branch from c938bcf to 3fbff88 Compare May 15, 2026 14:01

joelanford mentioned this pull request May 17, 2026

Remove e2e-test-registry image build from operator-controller main config openshift/release#78581

Open

2 tasks

tmshort force-pushed the fix-experimental-e2e branch from 3fbff88 to c87aca2 Compare May 18, 2026 15:24

tmshort force-pushed the fix-experimental-e2e branch from c87aca2 to d1f5659 Compare May 19, 2026 02:26

tmshort changed the title ~~UPSTREAM: <carry>: Fix flags passed to test-experimental-e2e~~ OPRUN-4607: Fix flags passed to test-experimental-e2e May 19, 2026

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 19, 2026

openshift-ci Bot assigned dtfranz May 19, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label May 19, 2026

Conversation

tmshort commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

openshift-ci Bot commented May 11, 2026

Uh oh!

tmshort commented May 11, 2026

Uh oh!

tmshort commented May 11, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tmshort commented May 11, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tmshort commented May 12, 2026

Uh oh!

tmshort commented May 12, 2026

Uh oh!

tmshort commented May 12, 2026

Uh oh!

tmshort commented May 13, 2026

Uh oh!

tmshort commented May 13, 2026

Uh oh!

tmshort commented May 14, 2026

Uh oh!

tmshort commented May 14, 2026

Uh oh!

tmshort commented May 15, 2026

Uh oh!

openshift-ci-robot commented May 19, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

dtfranz commented May 19, 2026

Uh oh!

openshift-ci Bot commented May 19, 2026

Uh oh!

dtfranz commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tmshort commented May 11, 2026 •

edited

Loading

coderabbitai Bot commented May 11, 2026 •

edited

Loading

openshift-ci-robot commented May 19, 2026 •

edited by openshift-ci Bot

Loading

dtfranz commented May 19, 2026 •

edited

Loading